Minimum Variational Stochastic Complexity and Average Generalization Error in Latent Variable Models

Bayesian learning is often accomplished with approximation schemes because it requires intractable computation of the posterior distributions. In this paper, focusing on the approximation scheme, variational Bayes method, we investigate the relationship between the asymptotic behavior of variational stochastic complexity or free energy, which is the objective function to be minimized by variational Bayes, and the generalization ability of the variational Bayes approach. We show an inequality which implies a relationship between the minimum variational stochastic complexity and the generalization error of the approximate predictive distribution. This relationship is also examined by a numerical experiment.